Abstract

Background: There is increased pressure to conduct systematic reviews as comprehensively as possible within the constraints of a timescale that meets the needs of policy makers. Objectives: We aimed to evaluate how the findings and conclusions of a full Cochrane systematic review differ when certain aspects of the methodology are limited in order to conduct a more rapid systematic synthesis of the evidence. Methods: Blinded to the results of a Cochrane systematic review on the effectiveness of directly observed therapy for treating tuberculosis, we limited three key features of the Cochrane review protocol. 1) The literature search: using the same key words as those used in the full Cochrane review, we reduced the number of databases searched; 2) study selection: using the same inclusion criteria as the Cochrane review, we used only one researcher to select studies; and 3) data collection and synthesis: using the same data collection and synthesis procedures to meta-analyse the data, we used only one researcher to extract the data. Any queries regarding study selection and data collection were resolved with a second researcher. Results: The results compare how limiting certain aspects of the methodology of a full systematic review impact on: 1) the studies identified from the search; 2) the studies included into the review after screening; and 3) the results of the meta-analysis after data collection. Conclusions: The results of this more rapid systematic synthesis of data are compared with those of the Cochrane review to understand how limiting methodological aspects impact on the conclusions which can be drawn for policy makers. Based on these findings, the benefits and limitations of each approach are discussed, as are the limitations of the study.
Hide

Abstract

Vlachojannis et al reported a systematic review on the medicinal use of potato-derived products. The authors identified five trials for inclusion in the review, including one study on the treatment of burns. Based on this RCT the review authors concluded that potato peel is not recommended for burns. As the authors of a rapid review on the use of potato peels for burns, we read this systematic review with great interest. Although the concept of rapid review is rising, accelerating the review process might introduce bias and its conclusions may be subject to change once a systematic review is available. Since this rapid and systematic review were done at similar times, we explored if the results were consistent. We identified three trials on the use of potato peels. Two of these trials were not mentioned in the systematic review. The evidence indicates that sterile potato peel dressings are better than gauze alone during the healing phase.While there is no evidence of an antibacterial effect, we concluded that potato peels promote healing. Potato peel dressings might be the best available dressing in resource poor countries. Because systematic reviews have a major impact it is crucial that systematic reviews meet specified quality criteria. Therefore we draw attention to adherance to the PRISMA statement.
Hide

Abstract

INTRODUCTION: Rapid reviews are being produced with greater frequency by health technology assessment (HTA) agencies in response to increased pressure from end-user clinicians and policy-makers for rapid, evidence-based advice on health-care technologies. This comparative study examines the differences in methodologies and essential conclusions between rapid and full reviews on the same topic, with the aim of determining the validity of rapid reviews in the clinical context and making recommendations for their future application. METHODS: Rapid reviews were located by Internet searching of international HTA agency websites, with any ambiguities resolved by further communication with the agencies. Comparator full systematic reviews were identified using the University of York Centre for Reviews and Dissemination HTA database. Data on a number of review components were extracted using standardized data extraction tables, then analysed and reported narratively. RESULTS: Axiomatic differences between all the rapid and full reviews were identified; however, the essential conclusions of the rapid and full reviews did not differ extensively across the topics. For each of the four topics examined, it was clear that the scope of the rapid reviews was substantially narrower than that of full reviews. The methodology underpinning the rapid reviews was often inadequately described. CONCLUSIONS: Rapid reviews do not adhere to any single validated methodology. They frequently provide adequate advice on which to base clinical and policy decisions; however, their scope is limited, which may compromise their appropriateness for evaluating technologies in certain circumstances.
Hide

Abstract

Executive Summary ObjectivesThe objectives of this report were:* to assess current practice in the preparation of rapid reviews by HTA organisations nationally and internationally,* to examine the current evidence base pertaining to the methodology of rapid reviews and*to determine if there are differences in the essential conclusions of rapid and full reviews on the same topic.
Hide

Abstract

OBJECTIVE: To describe a method for the rapid appraisal of new interventional procedures and to compare its conclusions with those derived from a slower, more thorough method. METHODS: Explanation of an algorithm, pragmatically developed over a decade at the British United Provident Association (BUPA), to classify requests for funding for new interventional procedures as 'Fund routinely'; 'Fund as a one-off'; 'Fund in trial only'; 'Do not fund currently' within about 48 hours. Comparison of the resulting categorizations of 39 interventional procedures against the subsequent work of the English National Institute for Health and Clinical Excellence (NICE) Interventional Procedures team. The first two BUPA categories were equated with NICE's 'evidence adequate' and the second two with 'evidence inadequate'. RESULTS: The algorithm is fit for purpose. It facilitated 114 requests for funding, received before June 2005, being successfully allocated: fund routinely, 33 (28.9%); fund as a one-off, 20 (17.5%); fund in trial only, 37 (32.5%); do not fund, 24 (21.1%). NICE subsequently categorized 18 being 'evidence adequate' and 21 'evidence inadequate'. There was concordance between BUPA and NICE on 35/39 (90%) of the topics. The four discrepancies are discussed. CONCLUSION: Rapid appraisal of new interventional procedures using the BUPA algorithm is feasible and in most instances its output is similar to that obtained from a slower more thorough method.
Hide

Abstract

Background:Increasingly, healthcare decision makers demand quality evidence in a short timeframe to support urgent and emergent decisions related to procurement, clinical practice, and policy. Health technology assessment (HTA) producers are responding by developing innovative approaches to evidence synthesis that can be executed more quickly than traditional systematic review. These approaches, and the broader implications they bring to bear on health decision making and policy development, however, are generally neither well-understood nor well-described. This study intends to contribute to an emerging literature around methodological approaches to rapid review in HTA by outlining those developed and implemented by the Canadian Agency for Drugs and Technologies in Health (CADTH).Methods:Since 2005, CADTH has developed and implemented a rapid review approach that synthesizes evidence to support informed healthcare decisions and policy. Rapid Response reports are tailored to the identified needs of Canadian health decision makers, representing a range of options with regard to depth, breadth, and time-to-delivery.Results:Preliminary observations indicate that CADTH's approach to rapid evidence review is generally well-received by Canadian health decision makers; real-world case studies provide pragmatic examples of how health decision makers have used Rapid Response reports to support evidence-informed health decisions across Canada.Conclusions:Rapid review is becoming an increasingly important approach to evidence synthesis, both within and external to the field of HTA. Transparent reporting of the methods used to develop rapid review products will be critical to the assessment of their relevance, utility and effects in a range of contexts.
Hide

Abstract

There is a tension between conducting comprehensive systematic reviews and completing them in time to meet policy-making deadlines. The 'rapid evidence assessment' has been proposed as a solution to this; offering rigorous reviews in a condensed timescale. While used frequently in healthcare, this mode of reviewing presents considerable challenges in social policy. We describe some potential problems and suggest reviewing strategies that can overcome some of them. There are situations, however, in which it may not be feasible to embark on a rapid review, and caution should be exercised when selecting this method.
Hide

Abstract

BACKGROUND:Rapid reviews have emerged as a streamlined approach to synthesizing evidence - typically for informing emergent decisions faced by decision makers in health care settings. Although there is growing use of rapid review 'methods', and proliferation of rapid review products, there is a dearth of published literature on rapid review methodology. This paper outlines our experience with rapidly producing, publishing and disseminating evidence summaries in the context of our Knowledge to Action (KTA) research program. METHODS:The KTA research program is a two-year project designed to develop and assess the impact of a regional knowledge infrastructure that supports evidence-informed decision making by regional managers and stakeholders. As part of this program, we have developed evidence summaries - our form of rapid review - which have come to be a flagship component of this project. Our eight-step approach for producing evidence summaries has been developed iteratively, based on evidence (where available), experience and knowledge user feedback. The aim of our evidence summary approach is to deliver quality evidence that is both timely and user-friendly. RESULTS:From November 2009 to March 2011 we have produced 11 evidence summaries on a diverse range of questions identified by our knowledge users. Topic areas have included questions of clinical effectiveness to questions on health systems and/or health services. Knowledge users have reported evidence summaries to be of high value in informing their decisions and initiatives. We continue to experiment with incorporating more of the established methods of systematic reviews, while maintaining our capacity to deliver a final product in a timely manner. CONCLUSIONS:The evolution of the KTA rapid review evidence summaries has been a positive one. We have developed an approach that appears to be addressing a need by knowledge users for timely, user-friendly, and trustworthy evidence and have transparently reported these methods here for the wider rapid review and scientific community.
Hide

Abstract

A brief review of evidence is limited in time and/or scope compared to a comprehensive review. However; brief reviews are important not only in meeting the needs of policy makers and practitioners, but also in providing students and researchers with an overview of the evidence. In this paper we summarise and evaluate alternative methods for brief reviews, including: using strict inclusion criteria; reviewing only a sample of evidence and eliminating or reducing steps in the review process. We examine a sample of brief reviews and found that the majority did not meet the methodological standards of comprehensive reviews. We conclude by recommending some methodological standards for brief reviews
Hide

Abstract

BACKGROUND:The systematic review is becoming an increasingly popular and established research method in public health. Obtaining systematic review skills are therefore becoming a common requirement for most public health researchers and practitioners. However, most researchers still remain apprehensive about conducting their first systematic review. This is often because an 'ideal' type of systematic review is promoted in the methods literature. METHODS:This brief guide is intended to help dispel these concerns by providing an accessible overview of a 'real' approach to conducting systematic reviews. The guide draws upon an extensive practical experience of conducting various types of systematic reviews of complex social interventions. RESULTS:The paper discusses what a systematic review is and how definitions vary. It describes the stages of a review in simple terms. It then draws on case study reviews to reflect on five key practical aspects of the conduct of the method, outlining debates and potential ways to make the method shorter and smarter--enhancing the speed of production of systematic reviews and reducing labour intensity while still maintaining high methodological standards. CONCLUSION:There are clear advantages in conducting the high quality pragmatic reviews that this guide has described: (1) time and labour resources are saved; (2) it enables reviewers to inform or respond to developments in policy and practice in a timelier manner; and (3) it encourages researchers to conduct systematic reviews before embarking on primary research. Well-conducted systematic reviews remain a valuable part of the public health methodological tool box.
Hide

Abstract

BACKGROUND AND OBJECTIVES: The expansion of evidence-based practice across sectors has lead to an increasing variety of review types. However, the diversity of terminology used means that the full potential of these review types may be lost amongst a confusion of indistinct and misapplied terms. The objective of this study is to provide descriptive insight into the most common types of reviews, with illustrative examples from health and health information domains.  METHODS: Following scoping searches, an examination was made of the vocabulary associated with the literature of review and synthesis (literary warrant). A simple analytical framework -- Search, AppraisaL, Synthesis and Analysis (SALSA) -- was used to examine the main review types.  RESULTS:Fourteen review types and associated methodologies were analysed against the SALSA framework, illustrating the inputs and processes of each review type. A description of the key characteristics is given, together with perceived strengths and weaknesses. A limited number of review types are currently utilized within the health information domain. CONCLUSIONS: Few review types possess prescribed and explicit methodologies and many fall short of being mutually exclusive. Notwithstanding such limitations, this typology provides a valuable reference point for those commissioning, conducting, supporting or interpreting reviews, both within health information and the wider health care domain.
Hide

Abstract

OBJECTIVES: The aim of this study was to explore the needs and requirements of decision makers in our regional healthcare system for health technology assessment (HTA) products to support portfolio development planning for a new HTA agency in Madrid, Spain. METHODS: A Delphi study was conducted during 2003. Questionnaires were developed based on a review of products and services offered by other agency members of the International Network of Agencies for Health Technology Assessment, and included preference and prioritization questions to evaluate twenty-two different products and services. The initial Delphi panel involved eighty-seven experts from twenty-one public hospitals, eleven primary healthcare centers, six private hospitals, and eight departments of the Regional Ministry of Health of the Community of Madrid. RESULTS: The global participation rate was 83.9 percent. Ten of the twenty-two possible products were rated of high interest by more than 80 percent of respondents. Important differences in preferences and priorities were detected across different settings. Public hospitals and primary healthcare centers shared a more "micro" perspective, preferring classic technology-centered HTA products, whereas private hospitals and Ministry representatives demanded more "macro" products and services such as organizational model and information system assessments. CONCLUSIONS: The high participation rate supports the representativeness of the results for our regional context. The strategic development of an HTA portfolio based on decision makers' needs and requirements as identified in this type of exercise should help achieve a better impact on policy development and decision making.
Hide

Abstract

OBJECTIVES: To elucidate important differences between a health technology assessment (HTA) and a systematic review, using an HTA of positron emission tomography (PET) as an example. METHODS: Interviews with seventeen individuals who were authors or users of the PET HTA. RESULTS: Those interviewed identified seven areas in which HTAs often differ from traditional systematic reviews: (i) methodological standards (HTAs may include literature of relatively poor methodological quality if a topic is of importance to decision-makers), (ii) replication of previous studies (relatively common for HTAs but not systematic reviews), (iii) choice of topics (more policy oriented for HTAs, while systematic reviews tend to be driven by researcher interest), (iv) inclusion of content experts and policy-makers as authors (policy-makers more likely to be included in HTAs, although there are potential conflicts of interest), (v) inclusion of economic evaluations (more often with HTAs, although economic evaluations based upon poor clinical data may not be useful), (vi) making policy recommendations (more likely with HTAs, although this must be done with caution), and (vii) dissemination of the report (more often actively done for HTAs). CONCLUSIONS: This case study of an HTA of PET scanning confirms that HTAs are a bridge between science and policy and require a balance between the ideals of scientific rigor and the realities of policy making.
Hide